Learning Underlying Forms With MaxEnt
نویسندگان
چکیده
It has been shown previously that a Maximum Entropy model can be used to learn the correct weights on linguistic constraints in the supervised case [4]. In this paper, we show how to extend the existing work to learn underlying forms in addition to linguistic constraints, discussing the practical details of the priors and the constraint sets that allow this to work. In addition, we discuss a technique to factor variables out of the loss function, thus allowing for more efficient computation of an exponential number of possibilities. Also included is a discussion of how to extend the results to handle sampling.
منابع مشابه
Semi-Supervised Learning via Generalized Maximum Entropy
Various supervised inference methods can be analyzed as convex duals of the generalized maximum entropy (MaxEnt) framework. Generalized MaxEnt aims to find a distribution that maximizes an entropy function while respecting prior information represented as potential functions in miscellaneous forms of constraints and/or penalties. We extend this framework to semi-supervised learning by incorpora...
متن کاملStructural Maxent Models
We present a new class of density estimation models, Structural Maxent models, with feature functions selected from a union of possibly very complex sub-families and yet benefiting from strong learning guarantees. The design of our models is based on a new principle supported by uniform convergence bounds and taking into consideration the complexity of the different sub-families composing the f...
متن کاملMaximum Entropy Semi-Supervised Inverse Reinforcement Learning
A popular approach to apprenticeship learning (AL) is to formulate it as an inverse reinforcement learning (IRL) problem. The MaxEnt-IRL algorithm successfully integrates the maximum entropy principle into IRL and unlike its predecessors, it resolves the ambiguity arising from the fact that a possibly large number of policies could match the expert’s behavior. In this paper, we study an AL sett...
متن کامل1 A Priori MaxEnt H ( S ) independent class analysis ( ica ) vs
Two mirror symmetric versions of the maximum entropy (MaxEnt) methodology are introduced and compared: (1) A posteriori MaxEnt Independent Component Analysis (ICA) H(V) was proposed by Bell, Sejnowski, Amari, Oja (BSAO) (early by Jutten & Herault, Comon and Cardoso (JHCC) in France). It is ambitious to factorize the unknown joint-probability density function (j-pdf) using the post processing al...
متن کاملFast Inverse Reinforcement Learning with Interval Consistent Graph for Driving Behavior Prediction
Maximum entropy inverse reinforcement learning (MaxEnt IRL) is an effective approach for learning the underlying rewards of demonstrated human behavior, while it is intractable in high-dimensional state space due to the exponential growth of calculation cost. In recent years, a few works on approximating MaxEnt IRL in large state spaces by graphs provide successful results, however, types of st...
متن کامل